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ABSTRACT 

Part I summarizes data on the final version of the 
Student. Self -Report, an affective, multiple-chioce questionnaire 
developed to measure thetkinds of behavioral and attitudinal changes 
which elementary counselors most often attempt to produce when 
providing services to students in Grades 3-6. Part I contains 
reliability and validity data f6r the instrument, a model of - 
affective development resulting from a path analysis done on these 
data, a measure of the traits exhibited by students referred for 
counseling services, anda measure of the effects of counseling 
services. The measure' does 'not consistently show gains for students 
receiving counseling services. Nevertheless the data provide insight 
into the development of affective problems in elementary children and 
the way in which counseling programs address these concerns. A 
parallel- instrument was developed to measure similar affective 
content tox children in Grades 1-2. Part II presents validity and • 
reliability data collected* in^developing a pilot version of the , 
Student Self-Repeat. In general, results suggest that this instriiment 
measures the same attitudes and behaviors measured, by the upper level 
instrument. However, the instrument is difficult for primary children 
and invalid results are produced for many of these students. 
(Author/AL) : 



* * * * * *r* ************************************************* *,* ************* 

t * . 1 Reproductidns supplied. by EDRS are the best that Can be made * 

*• from the original document. * * 

V*******^^*^^*******^*************************************************^ 



* -4 \ 

U-S DEPARTMENT OF EDUCATION 
NATIONAL INSTITUTE OF EDUCATION 

EOUCATIONAI RESOURCES information 
4 CENTER (ERIC) 

• * This docom*?o| has been reproduce 

ff^cntved from ihe person or ofgjmijtson 

Mfftyr thjo^es have been fT*ade to tmp'ovc 
-.^4* reproduction quality. 

^ i | ¥ • Po^ns of vew of opfnsons stated >n this docw 

frtefil do npt ne\.eS5*Joiy f*rpreyrnt oHicvii NlE 
« ' - pOSit*>fi Of po! cy 



a 



RESEARCH INTO THE ")EVELOP ,, F > JT OF 
AFFECTIVE I!!STRU M EMTS I!! THE - 
*G SCHOOL DISTRICT 



Funded by the Mational Institute of Education 
GRANT WM5EF NIE-G-Sn-fy^ 1 



FINAL REPORT * 
PART 1 ' 



The^ Development of an Instrument to 
m Measure the Effect of Counseling Services 
on El*nsntary Children in Grades Three Though Six 



. *ay 1ft f 19^1 



s o 

v9 Proje ct Administrators » Proieot W^ac^r 

- — — 

^ Orac* Iverson, Director Robert Young 

' Office of Evaluation Services Office of Fvaluatior. S^rvi^ps 

5* Doris V/alker, ' Administrator * Prepared by : 

.Office of Evaluation Services / ~ " 



Robert Young 

Office 4 of revaluation Services 
Ronda Hunter 

"Uchiqan Stats University 
John Hunter 

Michigan Stat* University ' 



1 



9 

ERIC 



. ~ PREFACE 

\ « ' 

\T . H 

This is Part 1 of the final renort of the MIE funded project, Research 
into the Developnent of Affective Instruments in the Lansing School 
- District. This project, funded from February 1S, iq^o _ February 
1981, provided monies to investigate the development of two affective 
measures. The instrument described j.n this ^eoort was designed to 
, measure the effects of counseling services on students 'in ^rad*s thr*e 

through six. The, other measure was designed for students in grades one 
and two and is described in Part 2 of this reoort. 

9 m 

This project was managed by 'Robert Young, Office of Evaluation 
Services, .Lansing School District* The administrator ultimately 
responsible f^r the project was Dr. Grace Iverson, director of the 
Office of Evaluation Services. Ms. Ronda Hunter and Or. John Hunter of 
Michigan State University were responsible for much of the research 
4| described .in this r^p-ort; however, the Office of Evaluation ^ervicbs is 

ultimately responsible fpr its contents. 
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Persqns wishing "more information should contact the Office of 
Evaluation Services at- (S17) yiU-nyn. 
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ABSTRACT 

The Student Self-Report is a ?S item, effective, ^ultiple choice 
questionnaire. .The instrument v was developed to 'leisure the kinds of 
behavioral and attitudinal changes v ./hich elementary jcounselors nost 
often attempt. to produae when they provide servic^. The instrument 
has three subscales measuring self-concept f classroom behavior and 
getting along with others. The instrument as a whole, and two of the 
sjubscales f self-concept and classroom behavior, have sufficient 
reliability and validity for use i& measuring differences ^between 
groups of students. The third subscale, getting along Vith others, 
includes items' of widely differing content and should not be used by 
itself to evaluate group differences. 

The instrument was administered to selected, students in nine elementary 
schools served by , counselor^ . A cluster analysis of the instrument was 
done* and the items were grouped 'into seven conter^clusters. \4 path 
analysis was also completed with these data. This analysis suggested a 
model for the development of* self -concept • 

Finally, the instrument was used to measure the impact of small ^Houd 
and individual counseling services. Significant effects were found for 
these services in increasing classroom compliance and positive 
attitudes toward making friends. These data were inconsistent, 
howfver, with the instrument's measure of progr^n change from the 
previous yesr*. This 'inconsistency suggested that the instrument shares 
a problem with other affective Pleasures we have' used; it is somewhat 
insensitive to program effect. Nevertheless , the results of the study 
provide insight into the .development of v af feet ive problems in 
elementary children and the way in which counseling programs address 
these concerrfs. 
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THE DEVELOPMENT OF T*?STRU?!ETT rn "c^usr •njj; rpprrj^ nc 
COU^SELTMff SERVICES 0*? ELEMS^RY SCHOOL CMTLDP^ 
1*1 GRAPES TH°EE THpnMGH SIX 



The research describe^ in this report reflects ef^rts by the Lansing 

Scool District* Office, of Evaluation Services, to develop affective 

instruments suitable for measuring the effects of elementary counseling 

services on students in grades three through six. These research 

activities were be^un because of difficulties we have encountered in 

measuring affective change. Prior to the development of She measure 

described in this paper, the Office of Evaluation Services had worked 

.with several "counseling tyoe" projects for which the objective data 

showed no change, while the staff directly involved with the project - 

counselors, teachers, oarents and building administrators argued that 

the orogram had produced major changes in students thaV the instruments 
f a 1 1 ed _ t.Q„ .measure * 

In the winter of 1077, the Lansinz School district received s soecial 
•allocation of funds from the M ichigan Legislature for providing 
counseling services to students in elementary school buildings. This 
grant included an evaluation connonent 'which reouired the 
identification or development of affective measures for evaluating th* 
program. A review of available instruments, however, revealed no 
instruments which were both technically sound f and also measured the 
types of changes counselors expected ty produce. For example, the 
Elementary School Test Evaluation M970") published by the Center for 
.the Study of Evaluation rates tests on the basis of measurement 
validity, examinee aopropriateness, administrative usability a n d norwd 
technical exellence. One hundred and ten aVfective instruments are 
listed for the first, third and fifth sradeK All but four of these 
tests were rated "poor" in the area of normecT technical excellence, *rk 
all but thirteen were rated "poor* in the area of measurement, validity . 

Although several technically sound scales are available, i.e., 
Cooper smiths (1975) Self-Esteem Inventory and the Pief-s -Harris " 
Children's Self-Concept Scale, these scales .tend to measure global, 
stable personality traits. Counselors, on the oth*r hand, orovide 
relatively short term intervention - we have found_£t to consist of 
five to twenty individual or small group sessions - and focus on rather 
specific problems which are interfering with the students 1 performance 
in school. Vfe found that most of th* counselors' time is sp*nt" in " 
responding to classroom problems, i.e., fighting, non-comoliance , *md * 1 
"attitude problems". Counselor's also stressed that they work on s<* f- 
concept and on helping student's build relationships with their dopps! 

We were not convinced that th* tyne of intervention provid4^j|py * • 
cbunselors is approoriateljr measured by general personality scales. 
Nevertheless, interviews with counselors suggested that observable, and 
therefore measurable, changes were reliably produced as a result of the 
counselors 1 activities. Therefore, a piloUstudont self-report 
instrument was developed to evaluate *the project. **e1 lability" and 
factor analysis data doll *o ted on this instrtfnent showed the instrument 



to be a stable,, valid measure which reflected behaviors! changes 
resulting from counseling services, fcfter this evaluation w*s 
completed, ^however, several coneys remained which are addressed by 
the results outlined in this oaoer. First, the data reported by Flood, 
Petersen and Yourii (19??>, offered only a %lob*\ m?asur~ of *>**?vy>rrl ' 
and attitudin*! changes, and did not pinooint t/ie specific ?r»~P^in 
which counselor intervention was most effective, ^econd , o»Jr * 
experience in evaluating the Counseling Project showed that the pilot 
instrument missed several important pffective *re*s =>ddres$ed by 
counselors. For example, the tfcst failed to measure student's positive 
sociaJ contacts with other stucfents *nd the extent to which students. 
ir<ed other people in their environment. Finally, th« factor analysis 
of the" pilot instrument failed to discriminate between several tyses of 
items which -seemingly tested different traits. 

This p^.per presents d*ta collected in our efforts to develoo an 
affective measure which addressed these- concerns, It* contains 
reliability and validity data for the fin-»l version of the Student *eCLf- 
Heport, a model of affective development resulting from a r*th 
analysis done on these data, a neasur* of the traits exhibited by 
students referr^ for counseling s*rvioss, and a measure of the effects 
of counseling services. V e found that the measure did not consistently 
show gains for students receiving counseling services, Nevertheless, 
the data provide valuable information on the develoonent of affective 
traits, the measurement of these traits f and the "way in which short 
term, school based, affective programs serve students with affective 
needs. 
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'Subjects 

The subjects for this study were third, fourth > fifth ^nd sixth <*r*de 
students, in nine schools in the Lansing School liistrict. Thes* schools 
fcad three ?rade configurations. Five of the % schools served students in 
f>rades K-*i f three' o^^he schools served .s^rflnt^ jfo ^inder^-rten and 
the fifth and sijJj hgj^and ope of the schools served students in 
grades K-3* In ■ a r*y ^ciodls one third *nd one fourth 

grade classroom greeted /for Jtestiw*. In the uop*r* 

elementary schools^^^^Bp^d W^ixth~gr3de classrooms w*re 
randomly selected fM BPr?^ In addition, counselors tested students 
who were served resu^f, but 1 who were not included in the classroom 
testing. Complete pre-test data were received for ^5 students, of 
these students had complete data for both the ore and t!j| post-test for 
at least on** of th* subscales. 



Instruments 

The primary instrument for this study was thu Student Self-Sepor 4 - , * 
•thirty-five question, multiole choice questionnaire. An expmole of a 

2 



question from the instrument is s^hown in Figure 1,fcelow. The entire 
-"Instrument is included in Anpendix- A. Each iten wa's scored as a Likert 
Scale,. The most positive of the % responses wa^ scored *s a ,f a»« ^ ^ nc j 
the least positive was scored as a f, 1 rt , 

( 

How many friends do you have 1 in school? 
A. I have a lot of friends* in school. 
•B.. I have quit* a few friends in school . 
C. I don't have very many friends in school. 
0. 5 don't have any friends in school. 

t Figure 1. An example of an item from the Student Self-Report. 

The Student Self-Reoort was develooed in the Lansing School District bv 
the Office of Evaluation Services to reflect "the affective issues most 
often addressed by counselors. The initial set of it^ms w*s developed 
after meetings in which counselors described the specific behnvior -nd ' 
attitude changes- oroduced by their services. A oilot instrument w»s 
developed and reviewed by the counselors. This instrument wps 
designed to adlress three general affective areas, sei f-cenceat , . 
classroom behavior and getting along with others. 

The pilot instrument was administered to students during the i^p.^q 
school year; reliability coefficients and factor analyses were 
completed or>*these data. Measured by Cronb*ch ! s Alpha, the 
reliability coefficient for the entire scale was The subsnale 

reliabilities were .7° for self-concept, .SO for classroom behavior and 
.59 for getting along with others. The factor analysed showed th*t 
most of the self-concept items were in one cluster, and most of th* 
classroom behavior items v/ere in another cluster. The getting along 
with others scale, however, contained thre* tynes of items. Some 
items asked the student whether other students liked them. These items 
correlated very well with the self-concept scale. A socond group of 
items did not correlate well with any, of the scales, but fit better 
with the classroom behavior items. The only remaining items correlated 
with each other, and seemed to measure belligerence, that is, hitting, v 
shoving and fighting. 

uteris which did not correlate well with any of the scales were r->mov°d 
from the instruments Redundant . items were also removed to shorten th<* 
instrument. The items on how well the student was liked wera moved to 
the self-concept scale. Finally, new items were written to attempt to 
measure getting along with others. These attempted to measure wh*th~r 
the student likes other students, and whether he or she Initiates 
positive social contacts with oth«r students. 

i/ith these revisions the size of the r inal instrument w*=s reduced from 
j J6 to 35 items. Reliability and validity H^t-.a ^re discuss^i in tV 
Result3 section of this oaper. 

'A second^ source of data for this study was weekly counselor loss. 
These logs listed the students seen by the counselors each week. ~he 
logs were used to determine the anbunt of counselor service r<*o>iv»d 
between the, pre and post-test by each of the t^s^ed students. 
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Testing Procedures 

Classroom testing was done by staff trained by the Offi.ee of Evaluation 
Services. The classroom teacher, however, renained'in the roon 'while 
testing was conducted. The tester introduced him or herself, expleihed 
the reasons we were conducting the testing, and explained that the 
tests were confidential , and that teachers, parents and counselors 
would not see the students 1 answers. After two practice questions,* the 
tester r^ad the test items to the students. The students also had 
their 04/n copy of the test. Counselors tested the students who had not 
been included in the random sample. Unless 'these students had a 
reading problem, they read and completed the test on their ,own*. 

Results and Disoussion 

A. 

Content Analysis of the Student Self-Report 

* 

The Student Self-Report contains twenty three items from the piloh ■ 
instrunent, and twelve new items. s It is organized into three sub- 
scales: self-concept, getting along With others and classroom 
behavior. 
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The sej/-ooncept subscale* is composed entirely of it*ms from the pilot 
instrument. Tn the analysis of that instrument, all of these items 
loaded on the sane factor. These items can he subcategorized , houever, 
ihto the following cont'ent areas: ; 

. 

Self-image : Items 2 t 3 t 5, 6 and 8 ' 

These items ask the students how well they l*ke Jshonselvtfs , how 
many things they do Veil, how good looking are they, Ad how often 
it is their fault when things go wrong. 

LjJgedL by others: Items 1, 1, 7, 9, ^0 and 11 
These itens probe whether the students oerceive themselves as 
likeable people. They" ask whether the student is easy to get 
along^with, whether the students believe other people When they 
t*ll then they like them, whether the student get blamed for things 
that are not -their fault; and whether the studerits feel they have 
friends and that other students like them. 

The getting along with others ^ubsc^le was largely rewritten aft^r the 
analysis of the pilot instrument. This analysis showed that the 
only items from the original subscale\which loaded on the same factor 
Vere items measuring antisocial behaviors - hitting, shoving, fi/g^ting,. 
etc. .The new subscale contains three items from the pilot instrument 
and eleven new items* These items can be subdivided into the following* 
content clusters: 

Belligerence: Items 12*, 1*4, 2^ and ?2 

These items ask the student how often, thejr fight, whether it is 
to wreck other people's stuff, whether they tease other children - 
and whether they get angry when the teacher points* out 4 a mistake. 
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Liking others: Items iy, 16, 17, 13 and ?j - «■ 

These items measure whether the, students li'Jb other students in 

' school, whether they feel the other students are nice peode, 
Whether they have*a good friend in class, and whether they would. 

,wan£ to workM/ith a group of other students from the class. 
** * ' 

Positive Social Skills: ; Items 15, 19 and 21 V _ ' . 

T , These items measure social skills which help students make and 
-keep friends: ^talking over* problems with friends, sharing antf 
joining other students in games. . 

• i 

Empathy: Items 2** and 25 

These items measure the extent to whtph tfie students try to help 
another student who is having problems: spending time with a 
student others *dislike,- asking a crying student what's wrongs 

All but one of the (flassroom behavior items had 'be*n on the 'allot - 
instrument and in the factor 'analysis of that instrument these items 
all loaded on the same factor. The items all address non-comoliance 
issues in the classroom; however, the content of the items suggest th 
following subclysters. ( 

Disobedience: Items 26, 2?, 29, 3? and 

These items measure whether the students do what they are told to 
do, take care of their boaks and materials, behave when the teach? 
is out of the room, never refuse to talk to *the teacher and nay 
attention in class. 

Talking: Itq/s.28, 31 and 32 

These items measure whether the students talk when they are not 
* supposed to, pother other students when they are trying to wor's, 
and are out of their seats when they are not supposed to be. 

Lack of -Concern for 'fork: Items *>0 and ?5 J 

These items asked the' students whether 'they complete their work, 

and whether they continue to work when the teacher isn't watching. 

The" preceding content .clusters Mve an overview -of the traits the 
subscales'were designed to measure. Self-concept was defined as both 
self image and an image of oneself as a person whom othersAike? 
Getting along with others was defined as a laJk of belligerence, the 
presence df some pro-social skills, a general attitude of regard* and 
liking of others and empathy for the problems of others. r inilly, 
the classroom behavior 'subscale measured* compliance issues - 
disobedience, talking out of turn and lack of concern for work. 
Clearly thefe is some overlaD in the content of &me of the clusters 
assigned to different subscales,' Somfe .of the itfems from the ,f liked by 
others" cluster seem to measure the same kinds of traits measured by 
the "1 iking, others" suhscale. The antisocial "behavior items from the 
getting along with others subscalp address issues which ar£ also- 
behavior problems in the classrcjon. Thus, these items might as easily 
ha.ve been placed in the classroom behavior suhscale, 
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The three subscales, however, are sot necessarily intended to be* 
orthogonal. The counselors indicated that Shese were the general aresL<, 
most often addressed , in individual counseling sessions, and. therefore , 
we attenpted to provide separate data for these three areas* «'e hoped 
that the discrimination between the tftree subscale scores would be ' 
sufficient to provide a more precise, measure than that provided by the 
total .scale score. Nevertheless *, we expected" that the three scales 
would be highly related, , ' , 

< 

Scale Reliability ! < I 

The primary question 'addressed by*this research was whether the Student 
Self-Report could be used as a reliable measure of self-concept , - 
classroom behavior, and getting aling 'with others* Therefore, two 
estamateg of reliability were computed for the Student Self-Report *nd 
its subscales. These data are shown in Table 1* 'Cronbach's Mpha v/as 
us _ed_a s_ J^easup.^.jLri t e ' r n al -^an^tenc y , and Pearson ^oduct^oment 
correlations we'r£ computed between yMa pre and" post-tests .to. provide a 
measure of t teSt-retsest reliability. Although the revised instrument 
contained fewer items, thest? data show that the reliability for' the 
Student Self-Report remained consistent with that of the pilot 
instrument. The reliability of th* ^e£tin£ aloSg with others subscale, 
however f remained lower' than that of the other subscales, suggesting * 
that the Revisions to this scale did not nroduae a unidinensionpl 
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TABLE 1 

Reliability Estimates^ for the Student Self-Report 
ancMts„Subsca1es 



SCALE 

4 


# OF 
■ ITEMS 


MEAN 


SO 


INTERNAL CONSISTENCY 
(CRONBACH'S ALPHAS 
PRE POST 


•TEST-RETEST 
(PEARSON'S r) 
PRE VS.-fOST 


Self -Concept \ 


11 


3.'11 


.42 


.73 


' .77 


.5'7 


* 

* Getting Along 
Witji/tfthers 


14 - 


3.24' 


.36 ' 


.64 


.70 

* 


.49 


Classroom Behavior 


. «o 1 


3.44 


A3 


.78 


,84 


.59 


Totai Scale 


36 


3.26 


.30 


.83 


.85 


.64 , ' 

* 






• 




s 


» * 


0 t 



The data, in Table ? show item statistics for the individual items of * . 
th * Student S£lf-Fepo|rt . Three sots of statistics *re included: -item 
means and, standard deviations; 'correlations between the item score *nd 
the total rscpres on the test and subtest (the correlations were 
corrected for, the inclusion of the. item score); and the factor loadings 

' - S " ' 6 
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TABU 2* 

Item Statistics and Factor Loadings of Items From 
the Student Self-Report Pre-Test 



ITEM 



MEAN 



SD 



CORRECTED ITEM -* . 
TEST CORRELATIONS 
TOTAL-TEST SUB-SCALE 



FACTOR LOADINGS 



I 



JUL 



'HI 



, . 1 
2 

■ .. a •* • 

4- * 

'"• 9.- 
• 10 

II" 

AVERAGE 
SELF-CONCEPT 
" , ITEM • 
12 

r*v. - ■ 13 
f / 14 ' . 
• , 15 ' ■ 

16 * 

17 • 
\ ' .18 

19 
' 20 
21 , * 
22 

1 23 

24 • ' 
25 

"AVERAGE 
GETTING ALONG 
• WITH OTHERS 
ITEM . 
.26 
27 

28 . ' 

2a' 
, 30 
' 31 ' 

32 

33 

34 " 

:. 35 

V AVERAGE . 
CLASSROOM BEHAVIOR 
ITEM' 
- . AVERAG£ ' 
. TOTAL SCALE ' 
« "fTEM 



J* 



3.23 
'3.31 
3.07 
3.33- 
2.99 
,2.82 
2.60" 
3 '.22 
3.16 
3.46 
3.00 



3.1-1 



*3r 



3.14 
3.-29 
3.10 
2.91 
3.26 
3.26 
3.76 
2,54 
3.68 
2.89 
3.71 
3.09 
'3.56 
3.16 

3.24 



3.69 
3.41 
3.30 
3.74 
3.47 
3.39 
3.10 
3.36 
3.38 
3*. 61 

3.44 



3.26 



.77 
* .'74. 
.83 
.■69 
.85 
.81 
1.06 
.74 
.74 
.72 
.86- 

.80 

.89 
.65 
.99 
.79 
>69 
.75 
.59 

1.02' 
.66 

..88 
.64 
.98 
.64 
.83 

.79 x 



.53 
.78 
.77 
.55 
.67 
.79 
".91 
.91 
.75 
.60 

.73 



.77 



.44 
'.•32 
.-30 
..24. 
.26 
.33 
.31' 
.-28 
.38 
.37 
.36 

.33 

.24 

.28 

.37, 

.14 

,53 

.45 

.32 

.28 

.28 

.24 

.25 

.18 

.21 

.17 

.28 ' 



.41 
.44 
.34 
.38 
.34 
.44 
.45 
.22 
.43 
.44 

.39 



.33 



.33 

.36 

.•27 

730- 

.47 

.47 

.24 

.25 

.50 

.49 

.51 

.38 

.17 

.25' 

.31 

.20 

.38 

.37 

.23 




.31 
.31 
• 25, 
.18 
.26 
.27 

.27 



.55 
.54 
!45 
.37 
.32 
.57 
.57 
.24 
.47 
.54' 

.46 



.36 



.29 
.08 
.23* 
.08 
.03* 
.11 
.24* 
.29* 
.03 
.02 
-.02 

.13 

.35* 

.03 

.48* 

.08 

.23 

.24 

.03 
. .14 

.27 

.03 

.34* 
-.11 ' 

.10 
/•13 

,17 



.57* 
.58* 
.57* 
.32 
.33* 
,66* 
.68* 
'.29* 
, .48* 
.57* 

.51 



.14 



.35 
.37* 
'.21 
.39* 
.52* 
.54* 
^.19 
.26 
.63* 
.66* 
:64* 

.43 

-.02 
.35* 

-.01 ' 
.15 
.56* 
.36* 
.44* 
.11 

-.05 
.17 
.14 . 
.34* 
.06 

-.09 

.18 



-.01 
.01 
.00 
.07 
.19 
.06 
.08 
.05 
.18 
.16 

.08 



.23 



.15 
.13 
.07 
•-.08 
'-.15 
-.12 
. .12' 
-.13 
.05 
•*. .03 
;09 

.01 

.12 

.17 

.21 

.29* 

.19 

.24 

.15 

.35* 

.36* 

.37* 

.31 

.21 ' 

.33* 

.44* 

.27 



.19 
.22 
-.01 

.08 
.02 
-.02 
.05 
.09 
.01 

.10 



.14 
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of the itens in a principal^ component fsctor analysis with iteration 
and varinax rotatiojp. The nunber of Actors. in this analysis \?%s 
limited to three since only three fetors had Ei gen-values grsat^r 
thar* one (1) afty-^c gtatj iq. ' * * » 

These data suggest, several conclusions. *First, the items fron th* 
classroom behvior subscale clearly define a variable which can add to 
the information provided by the total scale score. All but one of 
these items loaded on factor 1, with- an average loading of 
Furthermore, the .average' correl at ion between £he subset!* scor* and th* 
individual, iten scores w£s . as compared , to a correlation of . 
^between the iten scores and £he total score. The .difference between- 
these correlations was* significant at the ,05 level, t dep (o)* = 2.% 9 
p <$Q5. / • ; - *• , * w ~\ - 

Th4 self-concept scale is less wsll defined, but still provides 
additional information compared io the total score. Eight .of th^ 
eleven itens frgm thjLs s-csLe loaded on* factor 2 r with an average 
loading of ,U3, The average correlation be.tween the subscale score and 
the individual item score was .38 as compared vithva'n average 
correlation of .33 betyeen the item scores 'and the total score. T.his 
difference approached, but did not reach one. tailed significance at the 
• P5.^J1 " t # -d«p (10) s 1.75, .1« < £ <.05. \ ^ 

The items jjrom tt*e getting along with others scale did not form a 
unitary f^ctor^ Five of the fourteen items showed a primary/ loading 
on Factor 2 with -the majority of the ' self-concepE^i terns. These items 
asked students about how well they Jiked other students in the school.^ 
Three of the items loaded on Factor 1 with* the classroom behavior 
items; tl^ese items stressed belligerence - hitting, shoving, fighting, 
etc. The regaining items loaded on Factor 3 and measured positive' 
social contacts - sharing, taking care'of other students, etc. 

Thusf, the getting -along with others scale^ices not provide additional 
information beyond the* information provided by the total score. Tn 
fact, the awrage correlation-between the individual items and the 
subscale is lower than the correlation between the items a-nd the total 
score. There 'does^seerri to 4 bfe a ¥ cluster of items in this scale, 
however, which can be separated from the o^her>-items in the instrument; 
thesj^ items measure "pro social" behavior. 

/In general, the Student Self-Reoort$provides an affective measure 
with sufficient reliability to be used to evaluate affecti ve* o^ogrms . ' 
In addition, two of its subscales, the self-concept and classroom 
behavior scales provide reliable, ^more sfecific affective data. The 
third scale, getting along with others isOtess reliable, and does not 
?eem to provide information which differs systematically from the -total 
test score, ^ ' ' 

Fine Grain Cluster Analysis , . 

Th# factor analysis in the preceding section r^v^Ied problems with 
the content analysis. Ho single general factor was found for thp * 
getting along with others subscale, yet the analysis did not separate 
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items of clearly different content in 'other subscales, i.e., the self- 
image and lil^ng others subclustprs of items. - • • 

Therefore f % the items we're clarified according to content, based on a 
seri.es of 'exploration factor analysis, and this new content analysis 
was used to start a 3eries of confirmatory factor analyses. This 
process was 'continued until a final analysis emerged an C/hich* the 
content analysis and factor analysis were in oerfect agreement. 

In the final analysis, 27 items were categorized in- nine ^aningful 
clusters, ^while eight, items were placed. in a re.sidual category. The 

. residual category, it-ems seemed to measure different tr?its for 
different students.. For example, item 12 asks the student whether or' * 
not he/she <*ets angry when the teacher points out a mis^ke.* This was t 
originally , categorized in the, belligerence subcluster. The da t^ 
analysis showed that some stadents reacted as predicted (on the^sis 
of -whether or riot they get mad at the teacher); however many oth^r 
students seemed to react on the basis of whether they get mad at 
themselves if they made a mistake. Most items in the residual set are 
of* this sort. The items were open ta several interpretations or 
measured several variables. # None of these items correlated very v*ll 

^with eaoh^ot^er or any of the clusters- in the analysis. 



Table 3 shows the correlations between the nine content areas for thl* 
595 complete pre-tests. The correlations were computed ijsin* 
confirmatory factor analysis with" conrm^palities , and are corrected for 



attenuation . 




Table 3. Correlations between the fine grained clusters 5oc 
pre-tests,, corn 
omitted) 



Social Skills 
Belligerence 
'Talking 
Oi sobedient 

Lack -of concern for work- 
Personability 
Eriends 
%>oearnnce 
Jfelf-Liking: " 



tions 


corrected- for 


sitenuption , de 


cimal 


SS 


* 

8 


1 


D 


VJ 


P 


F 


A. 


100 * 


6? 




HP. 


' 3« 


a* 


?.n 


5 


57 , 


100 


62 


' 71 


«3 


*6 


. 0 


J7 


27 


62" 


.100 


69 


6« 


^8 


16 ' 


13 




71 


69 


foo 


75 


a 9 


16 


7 




HZ 


6«' 


75 


ion 


no 




?? 


U6 


36 




hq 


' M9 


100 




?fl 


?.n 


0 


16 


16 


•>Q 

J ■ 


' 6o 


100 


65 


5 


_7 


1P 


7 


28 


38 


65 


100 


8 


-i| 


1 1 


26 


51 


70 


7Q 


70 



SL 

-« 

11 

P6 

51 
7 n 

70 
7P 
100 



The preceding cluster analysis serves to validate the content analysi^ 
of the instrument, Ther*n are -several concerns,, however, with usin^ 
these clusters for measurement. Several of the clusters are very 
highly correlated and have essential identical (i.e., within sampling 
ejrror) cQ^r^latidns with all other variables. Therefore, thes* 
Qlyjsters can be combined to form more reliable clusters without losing 
essential information/ Moreover, no unique path dia^'fam can be 
*** 
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fitted to tjie clusters individually (Hunter and Oerbing, ln«prpss). 

Five of the fine grained clusters were combined psing these criteria 
into two second order clusters* Talking, disobedient and lack of 
concern Tor work w?*re combined "into a Mon Compliance cluster. The 
appearance and self-liking clusters were combined into a Self-In^ge 
cluster. While the -personabilit^ and friends, clusters were Eighty ' 
correlated, th'ey did not have paralLel .correlations with the other 
clusters, and therefore could not be combined.' " 

/tNthe end-of^the fine grain cluster analysis! P' 7 items were included 
in 'six meaningful global • clusters. These cluster*? included: 

Social Skills: Jtens ?U, 21, 25 and 1? * 

Jhis cluster included nost of the itens from the positive s^al* 
skills and empathy content clusters : i .e. £ "sharing; talking over 

-'problems with friends, spending time with- a student bthers 
dislike, asking a crying student what's ^wronp. 

Belligerence:, Items -go, '2? and ia . 

This -cluster included most of . t^ belligerency* items fron the 
content>analysis: i.e., wrecking other people's stuff, teasing 
other children, fighting, 
* « 

* Non-Compliance: "items 26, 27, 23 23, 30, ^1, 3?, ?H and 35 
This cluster included all. but one* of the stents from the classroom 
behavior subscale of the "instrument. 

* 

Personability : Itens 1, 16 and 17 " 

- These itens cane from the liking others and liked b'y others content 
analysis clusters. The questions seem to measure a positive 

.altitude towards making friends, I.e.", does the student see him or 
herself as hard to get along with end others as generally nice. 

Friends: Items 10, Q, 11 and 1S 

^Th^se items measure whether, the student ;fe41s liked by others, and 
includes most of the remaining items from the liking others and 
liked' by others content clusters* , 

- Self- Image: Items 2, a, 5 and 6 

This cluster' included most of the items from the self-image content 
c-luster. They measure how well th£ students like themselves, how 
many things they do well and how good looking theyfe^l they are. 

Descriptive statistics for the six global clusters are Shown in Table a 
for the pre-test n post-test and' change scores. • The means and standard 
deviations were calculated for average response rather than summed 
Response to preserve the original respojis*' Icale of 1 to 4. Two 
measures of reliability are Shown, Cronhaph's Alpha was' used to measure 
internal consistency, and Pearson Product - Moment correlations were 
computed between the pre and poa^test to estimate test-hetest 
reliability. ' ^ ^ " 

• , ■ % 
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Tabled. Descriptive statistics for the six global clusters on the pre 
and post-test (N = aus.) . . 



CLUSTER 


OF 

ITEMS 


«. x 


SD 


ALPHA 




POST-TEST 




CHANGE 


TEST-RETEST 
PRE VS. POST 


Social Skills 




3.04 


.52 


.49 


X 

2.94 


SO 
.59 


ALPHA 
.60 


-.10 


50 
.6*1 


ALPHA 
•26 


.40 


Mingerence 3 


3.49 


.55 


.57 


3.43 


.59 


.63 


-.07 


u .55 


.12 


.55 


Hon -compliance 


9 


3.45 


.*5 


.79 


3.25 


.54 ' 


.85 


.-.19 


.45 


.58 


.59 


Kersona&iiity 3 


3.25 


.56 


.62 


3.» 


..50 


.52 


-.03 


.56 


.23 


• 

.45 


FHtrids ' 4 


3.32 




.73 


3.35 


.53 


.72 


.01 


.54 t 


z 45 


.50 

> 


*eir-ijnage 


4 


3.12 


.52 


.61 4 


3.K 


^5i— 


.59 


loi 


< .50 


.17 


.52 



, These data suggest that a set of content clusters have, been identified 
which can provide more information than is prqvid*d_by the nore general 
subscales. These clusters have relatively good internal consistency 
reliability, Ir*addition, the pattern or correlations between the 
clusters* (shown ^n Table 5 of the next section) suggests that thes* 
clusters measure six different underlying variables since none of theke 
clusterfsrare parcel in their correlations with the othj>r clusters in 
the instrument. v - # » 

\ . 

Causal Analysis m * 

TaBle 5 presents the raw score correlations between the clusters on the 
pre'-tests, post-tests and the cluster change scores. These' 
correlations are strongly affected by error of measurement and are thus 
meaningful only' in terns of crude patterns. However, a consistent 
pattern can be observed withifi the cluster correlation matrices for the 
Dre-test. and post-test. That is, the lilghest correlations are'found, 
between adjacent clusters' in the matrix, and the correlations decrease 
with increased distance between clusters in the matrix, ■ The ability to 
develop a matrix with these properties suggests a rather simple causal 
hierarchy between the variables measured ^these clusters. In this 
model, shown in Figure 2, the relation 'between two non-adjacent >f 
clusters is mediated by variables which fail between them in the * \ 
matrix. \ 
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Figure 2. A causal model which fits the' relationship between the 
clusters of itsms on the Student Self-Report/ The correlations above ' 
the line taken were from pre-test data, those below the line were taken 
fron post-test data. The correlations were corrected for attenuation. 

Table 5. Raw score correlations between ore-test, p(5st^:est ?nd 
change scores (>I = /fU$ f test retest mrreHfttions 
underlined, decimals onitted. 
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-3 
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1 
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39 
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15 
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-29 
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-2 
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26 
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39 
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47 


33 


10 


16 


32 


45 


27 


21 


-12 
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-20 


-13 
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14 


•2 


14 


47 
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53 


2 


-1 


5 


32 


50 


41 


•10 


' \ 


-7 


-19 


-51 
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Self-fuiage 


7 


-3 


18 


33 


53 


100 
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-3 


11 


21 


34 


52 


-5 
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-4 


-14 


k 
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Low Belligerence 30 
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The model in Figure 2 would suggest, 'for example that increases in 
social skills would --a f feet compliance only to the extent that it 
reduced belligerence. Furthermore, the correlation between social 
skills and non-conpliance should aporoxlnately equal the product of the 
correlations between social skills- and belligerence and between 
belligerence and compliance. 



12 



This prediction is the basis fgr the fi^st test of the model, shown in 
Table 5 for the pre-test and Table 7 for the post-test. These tables 
show the actual correlations between the clusters in the first matrix. 
The* correlations between the clusters were then oredicted usinr; f"he 
nodel and entered into, the second matrix. For example, the entry for 
the predicted correlation between social skills and compliance was 
computed by^multiplying the intervening correlations in Figure 2V(*5 7 
and .70). The third matrix in these tables shows the deviation between 
the predicted correlations and the actual correlations. Mone of these 
deviations, for either the pre or the post-test, exceed the error 
expected from sampling error. Thus, no model can be develooed with 
these clusters to further explain the correlations between the clusters. 

Table 5. The error analysis of the static o?th model for the ore-test 
data using the model shown in Figure 2. (Correlations 
corrected for attenuation, decimals omitted.) 



Matrix 1. The 


pre-test 


correlations corrected 


for attenuation . 
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Matrix ? . Kfrors in fit (acutal ninus reproduced correlations). 





SS BEL COM 


PER 


FBI 
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Social Skills' 


, 0* 3 


25 


n 

X 


1 


Low Belligerence 


0* 


-2 




-2? 


Compliance 




o* 




-1 


Personability 








-1 


Friends 


• 






0* 



High' Self- Inage 

"*These errors are 0 since these correlations were used to^aredict 
the other cells in the matrix. 

Table 7. The error analysis of the static path nodel for the post-t 
data usinq the *nodel shown in figure 2. (Correlations 
corrected for attenuation* decimals or.itted.) 

Matrix 1 . The post-test correlations corrected for attenuation. 
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Matrix 2 . The reproduced correlations predicted by the oath nod 
in Figure 1b. 
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Matrix ?. The errors (actual -*ifrius reproduced correlations) 



*S5 BEL CO^' P^H PHI ST 

* 

Social Skills • 0* -13 2} -2 -10 

Low Belligerence n* iu -*?o -? n 

Compliance 0* -20 -9 

Personability n* m 

Friends « 0* 

High Self-Ima^e 

t 

*These errors are 0 since these correlations were use to predict 
the other cells in the natrix. 

The second test of this nod*l evaluates the directionality of the 
effects predicted by the nodel. This nodel oredJcts, for ^x^nole, that 
chafes in social skills' should affect belligerence, hut that changes 
in belligerence should not effect social skills. This sug^sts that* 
the correlation betw,e^ .social skills oh the pre-test and belligerence 
on the post-test should be higher than the correlation between 
tfelligerenoe on the pre-test and social skills on* the post-test. 

Table 3 presents the ,cross lag correla^ons between the pre-test *nd 
post-test clusters. Since the clusters \hsve been arranged in the 
hyothesized causal order, the correlations above the diagonal 
represent the relationship between the* pre-test score of eeach cluster 
and the post-test score of all clusters -higher in the causal 
hierarchy* The reverse is true for the correlations below th* 
diagonal. If the clusters have been cofrectly arranged, the 
correlation above the diagonal for a pair of clusters Should be l^r^er 
than the corresponding correlation below the diagonal. In 10 of th* 
15 conparisons, the predicted correlation was larger, in one conpariso.n 
the correlations were equal, and in the remaining four comparisons the • 
correlation below the diagonal was larger. The "lean of the 
correlations above the diagonal was .2^9 while the neap of the 
correlations below the diagonal was . ?5 C . This difference was 
significant at the .05 level using one tailed dependent t-test, t 
dep(14) - 2.06, p<.05, ' - 

Table Tire cross lag correlations between pre-test and post-test 

measures (corrected for attenuation, decimals omitted, test v 
retest correlations underlined). 







3 

t 


POST 

< 


-TEST 






\ 






I 

|s 


R EL 


COM 


PER 


pox 


ST 


p 


Social Skills 




5« 


33 


28 


\17 


-7 


p 


Lo<i Belligerence 


"6 


92 


53 


un 


' 2 


-7 


E 


Compliance ' 


32 


53 


?2 


uu 


30 


25 


T 


Personability 


15 


25 


an - 


79 


no 


75 


E 


Friends 


, 3 


-1 


6 


52 


50 


52 


S 
T 


High Self-image 


? 


-5 


15 


37 


51 


87 



O , K 15 

ERIC . 21 



Implications ,pf the Causal Analysis 

The oath diagram in Figure 2 has inDlications for therapy in that it 
predicts indirect effects. For example, if the counselor wants to 
^ improve a child f s self concept J then the path diagran suggests that one 
strategy would be to help the child nake friends. On the other hand, 
if the counselor tries to help a child L a arn social skills, then there 
will be a considerable lag before such a change works its way down the 
causal chain to self conceDt. Furthermore, there may be considerable 
attenuation in the impact by the time it moves through so nany 
intermediate stages. 

The path diagram also suggests th3t therapy may frequently prove 
ineffective; especially if directed at the last steps in 'the chain. 
For example, suppose that a child has a low self concept b#*caiise J-e/she 
is inconsiderate and hence unliked. If the counselor arranges for the 
child to form several new friends, then the path diagram predicts that 
there will be a concomitant improvement in self concept, However, the 
'pqth diagram *lso predicts that ther* v/iLl continue to be changes after 
the counselor has quit. The sane causal' processes which caused the 
child to be unliked will continue-to operate after therapy and are 
likely to result in a breech i^/the child's relations with his new 
friends and a consequent return to his original low level of self 
concept ♦ 

According to the path diagram, the pl^ce to elicit lasting 'change is to 
improve the child's social skills. ■ An increase in these skills will 
lead to a decrease in belligerence which will lead to an increase in 
compliance which will nake the chi,ld more* likeable which will lead to 
more friends and finally a h£ghfr self coneept* Of 6ourse, the fact 
that the model does not Contain causal antecedents of th^se social T 
skills doesnot^iean that there are none. It just means that they were 
ftotj tapped by this questionnaire . . 

Evaluating the Impact o f Counseling Services 

'" " " — ' ' * """* ' -Mil ■ 4 

Fach of the nine schools participating in this study received the.- full- 
time services of an elementary school counselor. The counselors 
maintained logs which listed the students who Were seen each week in 
individual or snail group sessions. These logs were used to deternine 
the amount of counseling service received by each of the students who 
were pre and post-tested with the Student Self-Report, 

As described in the methods section of this report, students were 
^tested Jin one of "two ways. Classrooms were randomly selected in each 
of the nine schools; students in these classrooms were pre and 'post- 
tested. In addition, at the tine of the pre-test counselors 
identifier) .students whom they expected to see regularly. These" 
students were pre and post-tested by the counselor if they had 'not he«n 
included in one of the randomly selected classrooms. - 

An examination of the amount of counseling services received by the 
randomly selected 'students showed that three district kinds of service 
patterns could be identified. A'najority of the students CO.**) 
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received no individual or snail group services, A second group of 
students, V**.Y 6f the population, received • these services, but only 
had one to five sessions. The%e, sessions tended to focus on acute 
behavioral problens in the school.- fighting, oroblens in i*he 
lunchroom, a one-tine problem, in the classroon , .etc . 

• * - "* * 

The Remaining students had fairly regu Var >meetin^s with the counselor 
over a period of two to nine months. Whil? these students reoresent a 
relatively snail percentage of the school p'opul^tion^ M5.*T> , they 
represent a major focuS of the conselor ! s proqr^r). - J e estinated fron 
the counselor logs that about 45?* of the counselors tine is spent with 
'these students (Young, 19°0). This was the reason we included the 
counselor tested students in the evaluation - ^e. needed 1 a* substantial 
sanple of these "high priority' 1 students who^ received r^ular 
counseling' services . • >r a 

Table 9 shotis Pearson Product-Moment Correlation between the nUnber of 
counseling sessions the students received' 2nd average ore-test, post- 
test and gain scores^on the Student Self-Peport ^nd its subsoales. 
Table 10 s|*6ws the relationship between the number- of sessions *nd the 
fine grain" clusters in the instrument. Vne correlations in Table 10 
were computed using a confirnatory factor analysis and- have been 
adjusted for attenuation due to error in the measures. 

* ■ 
negative correlations were found between the number of" sessions and 

each of the three pre-test subscales. These correlations >/ere highest 
for the classroom behavior and self 'concept scale. The .fine grafti 
clusters most 'highly correlated with the amount of service were # 
compliance, personability sn<j friends. These data surest that - 
students are nost often referred for behavioral issues and negative 
relationships with .other children. These data are in agreement with 
counselor los datq which showed that the majority of the high priority 
target students were referred for behavior problens. 

Table 9 shows that the only significant positive correlation between 
the amount of services* and change on t*\e instrument was found for the 
classroom behavior subscale. Table 10 shows that this service- h*d 'the 
most impact in the fine grain clusters of cono/Liance. and oersonability . 
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TABLE 9 



Pearson Produc^Honent Correlations 
Between Number of Counseling Session's 
and Pre-Test, Post-Test apd Gain Score 
for the Student Self-Report 





PRE-TEST- 


' POST-TEST 


QAPj 


Self-Concept 


-.20 


-.23 


-.03 




p=.00 


p=.00 


p=.?6 


Getting Along ' 


■ -.03 ■ 




.01 


With Others 


ps.O? 


"ps.O* ." 


p=.*J2 


Classroon 


- -.22 


-.11 


.09 


Behavior 


p=.on * 


pa. 01 




Total Scale 


-.21 • 


-.17 


.0? 




p=.00 . 


P=.0" ■ 


. P=.30 



^ • TABLE JO 

Relationship between MumbeV of Counseling Sessions 
and Pre-Test f Post-Test and Gain Scores 
for the Fine Grain Clusters 
of the" Student Self-Report 
, (Correl3tlohs Corrected for Attenuation) 





PRE-TEST 


POST-TEST 


GAP' 


Social Skills . 


.01 


-.03 


. 02 


Belligerence 


-.17 • 


-.15 


.On 


Compliance 


-.28 


-. n 


.19 


Personability 


-.33 


-.22 


.23 


Friends 


-.23 


-.2* , 


.01 


Self-Ina^e 


-.10 


-. 18 


-.1? 




Table 11 compares pre-test, post-test and gain scores for ^hicjh priorit 
students, those Students who received' at least five counselling sessions 
and comparison students, students who received no coun-seltng service* 
These 3ata sho^ 'that* significant differences were found between hi^h 
priority and comparison students on all thr;e<* of the frre-test measures; 
that is, high priority student scores indicated higher affective needs, 

18 ' ■ 



No significant differences were found, however, between gain scores for 
high priority and comparison students on any of the measures. , 

The data in Tables 9, 10 and 11 raise a concern about measuring orogr^m 
effects. , These data suggest thit the primary impact of counseling 
service was in the area of 'compliance and classroom behavior. Work -with 
the pilot instrument, ho'wever, had shovm significantly larger <*ains in 
self-concept for high priority target students. (Flood, Petersen, Youn^ 

Since only half of the self-concept items were retained on the 
Student* S£lf-Report, it was possible that we had eliminated the items 
which measured change on th* pilot instrument, However, we reanalyzed 
the^ data for the 1Q7S-79 program year using only the items retained on 
the" Student Self Report. This subset of items' showed more powerful 
'program effects than the original set;" the difference between m^an *rin 
scores was .11, F(1 t *IU6\ = **.305, p<.05. 1 . 



TA.BLE V1 ' " • 

Pre-Test, Post-Wt and Gain Scores 
of High Priori ty^id Comparison Students 

-fteT f -Concept ( 1 1 Items ) 

Hi?,h Priority Conp^ris.on \ynv\ 

Pretest 2.95 3.13' ■ F( 1,421) = 1?k35, p - 

Post-Test 2.93 " 3.1*5 *F(M21) = 21.55 p = .on 

Gain -.02 +.03 F(1,ii21) = 1.38, d = .2« 

*! 99 32 a 

Getting Along 'itth others 

High Priority Comparison A'.'OVA 

Pre-Test ^ 3.21 3.23 ■ F(1,'421> = 0.72, p = m y? 

Post-Test 3.12 3.17 F(1Ji21) = 1.?p p" = 

Pain -.09 -.<V$ F(1 f 4P1) = 0J40, n = .53 

N 101 322 . 



r 
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- ^ ; . , 

' \ 



Classroom behavior 



High Priority J Conpafrison v ANQVA - 

Pre-Test* 7 3.29 1 3,46 . F(l v «22> = 12.90..P = .on 

Post-Te>st . - J.n, 3.23 *F(l t U2*) = o = .00 



Gairr.. # -.09 - -.06 ' * , F( 1,422) = + 0.03, o 

H , ,101 32^ 



07 



Total Test 



n 



. ■ High Priority Comparison ; W^VA 

r * — " ■ 

.Pre-Test 3.15 3.2? F(1,«16) = 10. M9, p = .0£> 

4 

Post-Test ^ 3.05 3.2° F(1,m*} = lU.ag, p = .0 

Gain -.10 1.0? F(1* f ttl6) = n.04 f p = 

>! ■ 07 

These data suggest that the Student Self- p eport Shares a problem with 
other instruments we have used to measure affective chan<*4; it is 
somewhat insensitive to program effects. ( We 'are forced to conclude from 
the failure to replicate the findings of Flood, Petersen and* Young (1°??) 
that the improvement in self-concept measured ^in the first study was a 
data artifact rather than "real 1 * prograVi chnn^e. Given that otber 
mea'sui^s, i.e,, teacher comoleted checklists, reflected consistent 
improvement for the high priority students, (Flood, Petersen, Young,* 
1979; Young^, 1980) we feel that the failure to find change with this 
instrument is a problem with the instrument and not with the counseling 
program. , 



Summary ^nd Conclusions k 



This paper" has summarized data on the Student Self-Report. The reoort 
addresses the reliability of the instrument pnd its. subscales . Content 
and factor analysis data have been orpvided supporting the content ~" 
validity of the instrument," A causal hierarchy was developed for th* 
•instruments content clusters. Finally,, the instrument uas used to 
measure program change* 

The reliability data showed acceptably high internal consistency 
estimates of reliability for the total scale and two of the subscales; % K 
self-concept and classroom behavior. Test-retJest reliability was 
somewhat lower, but was sufficiently high .for use in evaluating program 
effects, #n exploratory factor analysis of the instrument suggested that 
the self-concept and classroom behavior items defined two scales which 
could provide mora specific affective information. y ' . t * 
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Confirmatory factor analysis 'procedures w[£re # used to examine the content 
analysis of the instrument. .A reasonably *ood nrateh was obtained between 
the content analysis anct the 'fine grained clusters identified in the 
instrument. Sight 'problematic items were identified which did not fit in 
any of the content clusters. v . 

The correlations between the fine grain clusters in the instrument were 
fit info a matrix in which the highest correlations were between adjacent 
flusters and correlations decreased with increased distance between 
clusters in the matrix. A hierarchical causal model for the development 
of self-concept was 'suggested by thiese data. Tests of thig ^oAnl ^irm>e 
run. These tests agreed with the ^predictions of the model, within the 
measurement limits of the instruments. * ' V 

Finally, the instrument was used to measure change produced by counseling 
intervention. These data showed that 'the instrument measured the kind of 
problems for whidh students are referred. However, only snail program * 
effects were identified. Since these pfTects were not consistent with 
the effects measured "in the program the previous year, we concluded that 
the instrument shares one of the problems we h*ve had with oVher 
affective evaluation instruments; it is not particularly sensitive to 
the, imgact of the program. 
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STUDENT SELf -REPORT 

Prepared 3y: 
.Patricia Flood 
Pat Petersen 
Robert Young 



PRACTICE - • 

*¥ " 

* m * 

Think about the color red. Row oftan do you like to weaf- red clothes 

A. r'almost always like to we*r red clothes. 

8- I usually Hke to- wear red clothes* 

C« I. usually don't like to wear red clothes. 

0. I almost never JHke to, wear red clothes. 

Imagine that you didn't have to go to school if ydu didn't want to. 
Oo you think that you^would.go to school even if you. didn't have to? 

A. Yes . ^ 

8. Probably a 1 • - 

/ C. Probably not / c 

0. Ho, no way % <> m 
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Name _ 
School 



ANSWERS UNTIL I HAVE REAP THEM ALL TO YOU — 



L Are you easy or hard to* get- along with? 

A. I'm almost always easy to get along with, 

B # I'm usually easy to get along with, 

C, Vm usually hard to get along with, 

0, I'm almost always hard to get alofig with, 



2. Sometimes we like ourselves a lot and sometimes we don't like 
ourselves very much at all. How do you feel about yourself most 

_ __of_ihfi._timeZ - 

A. I almost always like myself 

8, Host of the time I like myself, 

C. Most of the time ! don't like myself, 

0, I almost never like myself. 



3, 



When people tell ymi that they like you,* do you usually believe 
them or do you think they are just-sayfrtg that? 

A. I almost .always believe them, 
,8, I usually believe them. 
C. • I usually think they are just saying that, 
0. I almost always think they are just saying that. 



4. Think about *all the things that you do in school (school work, 
playing games, sports.,,,,}. How many things can you do well? 

A. I can do a lot of things well. 

8. I can do quite a few things well. 

C. I can't 'do very many things well, 

0. >l can't do anything well. 



5. How good-looking do you think you are? 

A*. I think I'm very good-looking, 

8. I think I'm pretty good-looking. 

C. I don't think I'm very good-lookingj 

0. I don't think I'm good-looking at. all. 
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How good-looking do you' think other kids think you are? 
Most of them think I'm very good-looking. 



A. 
8. 
C. 
0. 



Host of them think I'm pretty good-looking. 
Most of them think I'm not 'very good-looking. 
Most of them" thfHk. I'm not good-looking at all 



1 



fault? 



t an do people bl ap e y ou for things that really aren't your 



A. People almost always blame me for things that aren't my 
fault. J 

B. People usually blame me for things that aren't my fault. 
-C. People usually don't blame^me for things'that aren't my' 

fault. . 

0. People almost never blame ma for things that aren't my 
fault. 



8. Think about the times when something b^d hapoens or when things go 
wrong m school. How often js it your fault when thinfs go wrong? 

A. Itjs almost always my fault whan things go wrong. 

8. It s usually my fault when things go wrong. 

C. Sometimes it's my faujt when things go wrona. 

0. It s almost never my fault when things go wrong. 

9. Think about the students you knpw in your school. How many of the 
students that you know like you? 

A. Almost all of the students that I know like me 

8. Host of the students that I know like me. 

C. Host of the students I know don't like me. 

0.. Almost all of the students I know don't like me. 

* 

10. How many .friends do you have in school? 

A. I have a lot of friends in school.' 

8,. I have quite a few friends in school. 

4 'C. I don't have very many friends in school. 

0. I don't have any friends in school. 

11. Think about the times whan you want to' do things with other kids 
How easy is it for you to find kids # to do things with? 

A. It's almost always easy for me to find kids to do things wi>h 

8. Most of the time it's easy for me to find kids to do things with 

C. Most of the time it's hard for me to find kids to do things with 

0.- It's almost always hard for me to find kids to do things with. 
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mc NEXT F EW QUESTIONS ARE GOING TO ASK YOU A3QUT PEOPLE YQU K.1CW I?t 
SCHOOL, RE MEMBER. YOUR TEACHERS ANQ FRIENDS WILL NEVER SEE YOUR ANSWERS, 
SO PLEASE 8 £ AS HONEST AS YQU CAN, PLEASE WAIT 70 CIRCLE YOUR ANSWER 
UNTIL I'VE REAP THEM ALL TO YOU . 

12, Think about the times in school when you haven't done your work as 
well as you could have and your teacher points out y*ur mistakes. 
How often does it make you mad when your teacher. tells you that you 

. have made a mistake? 

A. It almost always makes me mad, 

8. It usually makes me mad. 

C. Sometimes it makes me mad. 

0. It almost never makes me mad. 

* * 

13, Think about the students you know in this school* How many of them „ 
do you like? 

A, I like almost aTl of the students I "know in this school. 
8, I like many of. the students I know in this school. * 
C, I don't like very many of the students I know in this school. 
0. I don't like any of .the students I know in this school. 

14, When other students get on your nerves, how often do you hit or shov* 
them? . 

A, I almost always hit or shov# them, 

" * 8, Most of the time L hit or shove them, 

C. Most of the time I doVt hit or shove them, 

0. I almost never hit or shove them. 
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Pretend there is a bunch of kids in the playground you don't know very 
well - playing something that you really like to play (like jacks, 
basketball-, tag, anything you really like). Would you ask to' o lay 
with them? 

A, Yes, for sure 
8, Yes, probably 
C. No, probably not, 
0, No, no way, 

15, How well do you get along with the other students in this class? 

A, I get along very well wltl^the other students in this class. 
3, I get^along pretty well with the other students in this class. 
C, I don't get along very well with the other students in this 

class. ^ 
0, I don't get along with the other students, iff cMs class at all. 

27 
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17. 



How many of the studants In this class are nice people? 



A, AlmosWll of them are nice, 

8. Many of them are nice* 

C. Not very many of them are nice* 

0. None of them are nice. 



18. Do you have a good friend in this class? 

A. Yes, I have a really gooii friend in this class, 

8. Yes, I have a kind of good friend in this class. 

C. No, I don't really have a good friend in this class. 

0. I have no good friend in this class at all. 



. When your best friend/^oes something that makes you mad, how often 
do you talk it over with him or her to work it out? 

A. ^$lmost always talk it over. 

8. I usually talk it over, / 
C. I usually don't talk it over. 
0. I never talk it over. 



20. Is it ever O.K. to wreck other people's stuff? 

A. Yes, it really doesn't matter. 

EU Yes, if they deserve it. 

C. Maybe, if they really deserve it. 

0. No, it's never O.K. *to wreck other oeooles stuff. 

y 

21. Pretend you have a special treat in your lunch. How often would 
share it with one or two other students? 

A. I would always share it. 

8. I would usually share it. 

C. I usually wouldn't share it. 

0. I almost never would share it. 



22. If you were on the playground at recess and you saw that one of the 
<ids rrom your class was crying about something, how often would you 
tease or make fun of them? 



A. 
8. 
C. 

0.. 



I would almbst always tease them. 

I would usually tease them. 

I would not 'usually tease them. 

I would almost ^ever tease them. 
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23*. If your teacher- assign yQu^$^$^r~tt&^zy$ -yatj-carr v/crrV cn it 
alone or in^a group, how often would you decide to work in a group? 

A- I would almost always decide to work in a group. 

3, I would usually de'cide to work in a group. 

C. I would usually decide not to work in a group. 

0. I would almost never decide to work in a group. # 

24. What if you were on the playground at recess and you saw chat one of 
the <ids from. your class was. crying or really uosat, would you ask 
them what's wrong? 

» 

A. I would almost always ask them what's wrong. ^ 
3. I would usually ask them what's wrong. 
C. I wouldn't usually ask them what's wrong. 
0. I would almost never ask them what's wrong. 

25. What if there were a new student in your class whom nobody liked 1 
very much and that student wanted to play with you and your friends 

at recess. Would it be okay? 

A. Yes, for sure. 
8, Yes, probably. 

• C. No, probably not. * 
0* No, no way. 

THE^NSXT FEW QUESTIONS ARE GOING TO ASK YOU ABOUT THINGS YOU 00 IN YOUR . < 

CLASSROOM. --REMEM8ER. YOUR TEACHERS WILL NEVER SEE YOUR ANSWERS, SO PLE&SS 
BE AS HONEST AS YOU CAN 8E. 

26, 



How often do you do what your teacher -tells you to do? 

A. I almost always do what my teacher tells me to do. 
8. I usually do what my teacher tells me to do. 

C. I usually don't do what my teacher tells me to do. 

D. I almost never do what my teacher tells me to do.-" 

■ 

27. Think about the times when your teacher has to be out of the classroom 
' *■ for awhile. How do you act when your teacher is out of the room? 

A. I almost alwayj^do /what I'm supposed to do when my teacher 

is out of the room* 
8. I usually do what I'm supposed to do when my teacher is out 
of the room. 

C. I usually don't do what Vm supposed to do when my teacher 

is out of the room. 
O.yi almost never do what I'm supposed to do when my teacher is 
* out of the room. 

, o k 29 ■ . ■ 
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-28?- How-bftefv-are yotrout-of-yourseat when you* re "not Tuppbse3~ so beT 

A. I am almost always out of jny seat when I'm not supoosed to be, 

8. I am often out of my seat when I'm not supposed to be. - 

C. Sometimes I am out of my seat when I'm not suoposed to be. ' 

- 0. I am almost never out of my seat when I'm not supposed to be. 



29, 
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How do you treat your 1 school books and other materials you use in the 
classroom? « , 

A, I almost always take good care of them. 

8* I usually take good care of th^m. 

C. I usually don't take good care of them. 

0* I almost never take good care of them. 



When 'yjur teacher gi 
IB" 



ves you school work to do, how often do you complete 



A. I almost always finish my school work. p 

8. I usually finish my school work. 

C. I usually don't finish my school work. ' . s 

0. I almost never finish my school work, 

31. Think about how you act i"n the classroom. How often do you bother other 
students when they are trying to wor"<? 

A. I almost always bother other students when they are trying to work. 
8. I usually bother other students when they are trying to work. 
C. I usually don't bother other students when they are trying to work.* 
0, I almost never bother other students when they are trying *to work. 

32. Kow often do you talk in class when you are not supposed to? t 

A. I almost always talk in class when I'm not supoosed to. 

8. I usualjy talk in class when I'm not supoosed to. 

C* I usually don't talk in class when I'm not supposed^to. 

0. I t almost never talk in class when I'm not supposed tp. 

33. Oo you ever refuse to talk to your teacher? 

A. Yes, a lot of the time I refuse to tal-k to my teacher. 

3. Yes, pretty often I refuse to talk to my teacher. 

C. Yes, sometimes I refuse to talk to my teacher. 

0. No, I never refuse to talk to my teacher. 



E 



3*. How difficult or easy is it for you to pay attention in class? 
A. It's almost "always easy for me to pay attention in class, 
8. It's usually easy for me to pay attention in class. 
C. It's usually hard for me to pay attention in class. 
0. It's almost always hard forme to pay attention in class. 

<^79 
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35. Think about those times when' the teacher gives you work to do bv your 
self. How often do you do the work-even if the teacher isn't watch inq 
you? x 3 



A. almost always do my work even if the teacher isn ' t watching me. 

B. I usually do my work even if the teacher isn't watchino ma 

C. I utually don't do my work unless the teacher is wacching me 
■ 1 almost never ^0 my work unless the teacher is watching me. 
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PREFACE 



This is Part 2 of the final report of the MIE funded project, Research 
into the Development of Affective Instruments in the Lansing School 
District. This project, funded from February 15, 1980 - February 14, 
19*1 » provided monies to investigate the development of two affective 
measures. The instrument described in this report was designed to 
measure the effects of counseling services on students in grades 3-6 
and is described in part 1 of this report. 

This project was managed by Robert Young, Office of Evaluation 
Services, Lansing School District. The administrator ultimately . 
responsible for the project was Dr. Grace Tverson, Director of the 
Office of, Evaluation Services. Ms. Ronda Hunter and Dr. John Hunter 
of Michigan State University were responsible for much of the research 
described in this report, however, the Office of Evaluation Services _i_s_ 
ultimately responsible for its contents. 



Persons wishing' more information should contact the Office of 
Evaluation Services at (517) 3714^113147, 



TABLE OF COHWlTS 



Preface . . . . ♦ # . 

List of Tables ... 

x • «• Iff 

Abstract ...... ' * - 

. . - ............ „ 

introduction < ........... . 1 

ffethods V . . . # * 2 

Results and Discussion y 

Basic TTem Statistics . T 3 

Content Analysis ' . . . . . 1 !" ! . ! 4 

Exploratory factor analysis for the full 'sample . . . . 5 

Confirmatory factor analysis for the full sample . . . . 7 

Identifying Response Biases* - . 8 

Exploratory factor analysis for the reduced sample . . 10 

/ Confirmatory factor analysis for the reduced samole . . . 10" 

Recomnendations for Instrument Improvement " \2 

.References * ♦ . . 13 

Appendix A - Instruments 14 

Appendix B - The item correlation - entire sample .... 22 

Appendix C - Confirmatory factor analysis - £4 

- : entire sample 24 

ApDendix D - The item 'correlation - reduced sample ! ! 26 

ApDendix E - Confirmatory factor analysis - . - 28 

* ^reduced sample . t 28 



ii 



ERIC 



40 



i « *«> * — -> «» 

) * « * » * — 

» ^ " ' * * ■*<& - 

. .* . ' *■ > • 

\ . ' l. .-'*-« ✓ L'IST OF TABUS , "-A 

■ * * * - * Page 

- v 

t ; TABLE 1; Basic item statistics for the primary level 

'student self-repbrt \ . . . , 0 , „ , /, , • / . *4 

* * • * . 

TABLE 2: ^Exploratory factor analysis and item direction • * * 

artifacts 6 

^ " «, * 

* TABLE 5: The factor correlation between the nairfsix 

clusters. for the entire sample . . . . 8f 

, TABLE U: Factor loadings apd iteh direction for the 

reduced sample* c ... ^ 10 

TABLE 5: Exploratory factor analysis for the reduced 

sample . . . *. 11 

* j TABLE, 6: The factor correlations for the refined clusters 

<* on the reduced sample • 12 

• * * 



1 



in 



41 



• " ABSTRACT ' ' 

* 

The primary level Student Sel£-Report is a 28 iten affective,' multiple 
choice questionnaire designed for use jiith first and second sra'ie 
students. The instrument was developed tb measure the kinds of 
behavioral and atf;itudinal changes counselors mos^Xoften attempt to 
produce when they orovicje services. > 

The instrument was administered. to UOS first and second parade students 
in stx elementary schools. We found th? instrument somewhat difficult 
for these students, and only 225 students completed all twenty eight 
items. .For these 225 students, the instrument had a reliability of 
.70," estimated by Cronbach's Alpha. 

*• , 
A content analysis of the instrument identified eight content clusters 

Social. Skills, Belligerence, Compliance, Friends, Self-Concept, Looks; 

Smart and Mood.. A confirmatory factor analysis. shewed that six of 

thebe clusters were generally internally and externally consistent; 
-however, two cft'the clusters, Looks and Smart, were elini-nate^ -and put 
4 into*a residual cluster. The cluster analysis was consistent with the 

clusters found in the Instrument for upper elementary students. 

The analysis also showed a disturbing pattern of response. The 
responses of at least ten* percent- of the 405. students- v/ere primarily 
determined by response bias (responding "always" or "never") rather 
than by item content. This pattern skewed the correlations between ' 
clusters of items so that positively scored items were more highly 
coggfilated than negatively scored items. 



Irt^g^ner^, *,he results suggested that the primary level instrument 
measures the same attitudes and behaviors -measured- by the uppervlev 



^instrument.. However, the instrument is rather difficult for students 
in the first and second ^rade, and invalid results are produced«^or ^a 
substantial number of 'these students. * ^ . 
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THE DEVELOPMENT OF V! TMSTR r *ME*'T TO "FJ SCIRE THE EFFECT^ OF* 
COUNSELING SEPVICES Of! ELEMEflTAUY SCHOOL CHIL^RF!! 
IN GRADES ONE AMD TVO 



The research described in this report reflects efforts by the Lansing 
Scool District, Office of Evaluation Services, to develop af Active 

'instruments suitable for measuring the effects of elementary counseling 
services oh students in grades one and two. These research activities 
were begun because of difficulties we hav0 encountered in measuring 

•affective change. Prior to th* development of the measure described in 
this paper, the Office of Evaluation Services had worked with several 
"counseling type" projects for which the objective data showed no 
change, while the staff directly invol^d with the project - 
counselors, teachers, parents and building administrators argued that 
the program had produced major changes in students that the instruments 
failed to measure. 

In the winter of 1977, the .Lansing School District received a special 
allocation of* funds from the Michigan Legislature for providing 
counseling services to students in elementary school buildings. Tnis 
grant included an evaluation component which required the 
identification or development of affective measures for, ev41uatin.<5 th° 
Drogram. A revjlewl of available instruments, however, revealed no 
instruments which i/ere both -technically sound, and also measured the 
types of changes counselors expected to produce. For example, the 
Elementary Sghool Test Evaluation (19 7 0) published by the Center for 
the Study ^if Evaluation, rates tests .on the basis of measurement 
validity, examinee appropriateness, administrative usability *nd norrned 
technical exeUence. One hundred and ten affective instruments are 
listed for the first, third and fifth grade. All but four of these 
tests were rated "poor" in the area of norrned technical, excellence, *nd 
all but thirteen were rated "poor" ij\ the area of measurement validity. 

Although several technically sound scales are available, i.e., ^ 
Coopersndth f s (19 7 6) Self-Esteem Inventory and the Pifrs Harris ' . 
Children's SelfVConcept Scale, these scales tend to measure ,£l<&al , 
stable personality traits. Counselors, on the other hand, provide 
relatively shor£ term intervention - we have found it to consist of 
five,, to- twenty individual or small group sessions - and focus on rather 
specific problems which ^are Interfering with the students 1 ' Derformance 
in school* We found that most of the counselors 1 time is spcint in 
responding to classroom problem^, i.e., fighting, non-coriDliance , and 
"attitude problems". Counselors also stressed that they work on se 1 f * 
concept and on helping students build relationships with their pe*rs. 

f /e were not ^convinced that the type of intervention provided by 
counselors is appropriately measured by general personality scales. 
Nevertheless, interviews with counselors suggested that ofcsarvahle, and 
therefore measurable, changes were reliably Droduced as a result of the 
counselors 1 activities. Therefore, a pilot student solf-renort 
instrument was developed to evaluate the project. ReliabUitySmri 
factor analysis data collected on this instrument showed the instrument 



to be a stable and valid measure. 

Since primary level children reprisent a major focus &f the counselors 1 
activities, we have attempted to develop a parallel instrument which 
could measure similar affective content for these children . This paper 
presents the reliability and validity datn collected in developing a 
pilot version of this instrument. 



METHODS 



Subjects 



The subjects for this study were 105* first and second ?rade students in 
six 'elementary schools in the Lansing School District. 

Testing was done in. the classroom. The classroom t^^iers were r*iven a 
copy of the test for review, and then asked if they would permit us to 
test their students. Testing was done in every classroom in which we 
received the teacher" s permission to test. 

r * ' - ' 

Instrument ; V J 



The instrument for this study was the Primary level Student Self- 
Report. The instrument was developed in the Lansing School District to 
parallel an affective questionnaire devetfcp^d for 8ld*r students. This 
questionnaire had been designed to measure specif ili^ehaviors and 
.attitudes addressed by counseling services. Thf^gfnstrument is a ?S 
question, multiple choice questionnaire. An example' of 2 question from 
the instrument is shown in Figure 1 below. The entire instrument is 
included in Appendix A. Each item had tj?ffee pdssible chQiees. The 
instrument was scored as a LiJcert Scale with t'he most positive ^response 
worth three points and the least positive response worth one ooint. 



How many of the kids in this cIpss li4fe you? Circle th* k box if 
all of them like you, the S triangle if some of them like you,' and 
the N circle if none of them like you. 

Figure 1. An example of an item from the Student Self-Peoort. 

Counselors were also interviewed 1 - to determine whether there were any 
additional affective issues whicff the instrument should m£asur£. As a 
result, items measuring emotional state were added to the instrument. 
After a draft <£f the instrument w^Jeveloped, all elementary 
counselors in the district were siverwj cooy of the inst.run^nt to* - 
review. Suggestions resulting from this reviejy w*re incoroorif.ed into 
the instrument. v 
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Testing Procedures 

Testing was done in the classroom by staff trained by the Office of 
Evaluation Services, The classroom teacher, however, remains in the 
room while the testing was conducted . The items were read to thf 
students, and students responded on the forn shown in Appendix A. 

RESULTS AMD DISCUSSIONS 

Basic Iten Statistics 
1 

Table 1 shows basic item statistics for the twenty-eight items of the 
Student Self->Report . These d^ta include item means and standard 
deviations, the number of students who answered the question, and the 
correlations between the item scores andtthe total te*st score. The 
items were all scored so that a 1 represented the nost negative 
response and a 3 represented the most positiv? response ; - 

The data in Table 1 show a substantial amount' of missing data, 
especially for the items at the beginning of the questionnaire. Most 
often, this occurred when students narked more than one response, 
suggesting ^hat'five or six practice questions should bedgA/eloped for 
the questionnaire. 

The average item score on the questionnaire was 2\?J\ with an average 
standard deviation of .72* The average correlation between the item 
scores and the total score was .3£". Several of. the items, however, 
notably items eight and twenty-three had substantially lower 
correlations. £ Cronbach f s Alpha/was computed as a measure of 4nt*rn?»l 
consistency for the 225 complete tests, alpha Equaled .?o. \J 





TABLE 1 

Basic Item Statistics * 
for the Primary Level Student Self-Report 
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The 23 items which ,nake up the present questionnaire are composed of 
two, sub.sets: 22 items constructed from the previous Questionnaire 
developed for older children and 6 mood i£ems to assess anxiety or 
depression. The content analysis of the items show 3 clusters or 
subscales. The following clusters were defined. 



/ 



. Social Skills: Items 19, 20, 21 and 22 
These items define positive social behavior - sharing, playing 
with a child your friends don*t like,' cheering bo your friands, 
doing something special "for your friends. « 

"Belligerence: Items 13 and 14 

These, items define anti-social behavior - hitting others, teasing 
others. s * 
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. Compliance: Items 15, 16 , 1? and 1ft 

These items defined acceptable classroom behavior - paying 
attention, not acting up, trying hard, behaving when the 
teacher leaves the room. 

Friends: It ens a, 3 and 10 

These items measure whether the student feel that others like him 
or her - do others like you, do others dislike you, would others 
want to sit next to you. j # 

* . * 

Self-Concept: Items 1 and 11 

These items measure whether - the student likes hin or herself - 
how often do they feel they can do everything well, do they like 
themselves. 

Looks: Items ? and 9 

These items were in the self-image cluster of the previous 
instrument and measure whether the child feels that hp or she is 
attractive . 

# 

Smart: Items 2* 5, 6, ? and 12 

These items were primarily new, but were intended as part of the 
self-image cluster. They measure whether the child feels good 
' about his ideas and ability to do^good work in class - how -of ten 
doeS the student feel dumb, proud of his or her work, do good 
work in class, feel like an important person in class, and fee] 
that hfs or her ideas are liked by others. 

Hood: Items 23, 24, 25, 26, 27 and 2H 

These items measure the emotional state, of^the child - how often 
the child feels frightened, lonely, scared, happy, excited, 
interested and that he or she is having fun. 

Ten of the items were negatively worded; that is a response of "always" 
indicated a Iflgative mood, attitude self-concept, etc. Jhe remaining 
18 items were positively worded. The negatively worded items were 
items 2, 8, 9, 11, 13, I*/ 16, 1*, 23 and 25. 

Upon re-examination, several of the item3 also seemed to have ambiguous 
content. Item 20 asked the student whether fie or she would play with a 
student his or her *Yiends didn't like. He assumed that a positive 
response would indicate positive social behaviors and* correlate with 
the social skills cluster. For students of this age, however, lo^rlty 
to ones friends may be more important. Item 23 asked the student if he 
or she is often lonely. We were unsure whether this would be a mood 
item or a friends item. 

, * 
Exploratory factor analysis for the full sample 

The exploratory factor analysis procedure used in this study was ( 1 ) 
Prihcipal axis factor analysis saving factors with an eigenvalue 
greatef than 1, (2) Varimax rotation, and (3) oblique multiple croups 
factor analysis using clusters defined by Wrigley's criterion (i.e., 
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placing each item irrthe group corresponding to its highest factor 
loading)* 

Table 2: Exploratory factor analysis and item direction artifacts. 

Table 2a , Factor loadings and item direction for the full set of 
405 children using pair wide deletion of missing data. 
Contingency table relating item direction to the high factor 
loading* ^ 

Item 

; Direction 

High + 
Factor I 17 q 

Loading TT 1 10 



Tabl,e 2b , Factor loadings and item direction for the 225 
children who answered all questions. Contingency table relating 
the item direction to th«* high factor loading for the item. 

Item * 
Direction 

High + 
* Factor I m 16 0 

/ Loading II 2 10 



, The "first correlation matrix analyzed was for the entire set of 
405 children (with pair wide deletion of missing data). Two factors 
had eigenvalues greater than 1. The pattern of loadings Mas quite 
striking and quite disturbing. The negatively worded items all had 
their highest loading on Factor II, while with one exception the 
positively worded* items all had their highest-' loading on Factor I. 
Moreover, the exceptional item was item 20 which was believed to be 
ambiguous on content grounds. The contingency table relating item 
direction to factor loading is shown in Table 2a. 

As described earlier in this paper* there were /iany missing responses 
*(66 missing out of 405 on item 1 for instance). Thus it was possible 
that the artifact of item, direction night have arisen from soma 
systematic pattern in leaving responses out. To check this, we. 
computed the correlation matrix for those 225 children who zns\ier a 'l 
every item. The results differed only slightly from those for the full 
set of 105 children. The contingency table showing the relation 
between item direction and factor loading is shown in Table 2b. Table 
2b shows that all 10 negative Items had their highest loading on Factor 
II r while 15 of 18 positive items had their highest loading on Factor 
I. Again, one of the exceptional items was the ambiguous item 2^: the 
other was iPenr 1. We concluded that the item direction artifact was 
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not due to patterning in the missing data. Indeed, inspection of the 
correlation matrix as a whale revealed only ninor differences from that 
for the whole sample. Thus there were no systematic differences 
between children who completed all items and children who omitted some 
items. 

Confirmatory factor analysis for the full sample - 

^The correlation matrix for the full sample was reordered to group the 
items into the clusters generated by the content analysis. The 
correlation matrix is presented in Appendix B. 

Six of the content clusters, social skills, belligerence, compliance, 
friends, self-concept* and moods, showed the expected strong cluster 
structure. Four bad items were identified/ The ambiguous item 20 
failed to correlate with anythirf£ aside from sampling error. ^ The sane 
was true for the ambiguous item 23-, although it is not clear, why it did 
not correlate with either of the possible clusters to which its content 
was related. Item * correlated negatively with the other friends. 
This may represent the item direction artifact in the data since item % 
was negatively worded and items U and 10 were positively worded. Item 
25 cor reT&t^d with no other item beyond samoling error. Again this may 
represent theMten direction artifact since item 25 was the only 
negatively worded mood item (other than ambiguous item 23). " 

Two clusters, looks and smart showed very low internal correlations. 
The two looks items were correlated -.01 with each othjr. This may 
have betfn partially due to the item direction artifact; item 3 is "good- 
looking" while item 9 is "ugly". The items from the. smart cluster were 
only slightly correlated with each other and were uncorrelated with any 
of the other clusters. There was no item direction artifact operating 
in this cluster, all the items from the smart cluster were positively 
worded. 

The correlation matrix for the whole sample subjected t 0 confirmatory 
factor analysis, i.e., oblique multiple groups factor analysis with 
communalities. The bad-items 23, 8, 2$ and 25 were dropped from their 
respective clusters and were added to a' residual cluster along with the 
items^from the looks and smart clusters. This confirmatory factor 
analysis is located in Appendix C. The confirmatory factor analysis 
showed that the six main clusters satisfied the confirmatory factor 
model to a considerable extent; the correlations were consistent within 
clusters and the items in each cluster tended to be parallel to each 
other. The primary deviations occurred within the compliance cluster 
where the positively and negatively horded items were not parallel. * 

Table 3 shows the correlations between factors from the confirmatory 

. factor analysis, These correlations were positive, but substantially 

* less than 1.00. Thus *the' clusters are clearly statistically 

distinguishable from one another. On the other hand, the effects of 
the item direction artifact are clearly visible in the table. Social 



Table 3* The factor correlations between the main six clusters for the, 
entire sample (decimals omitted, M c 
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skills, friends and mood are conposed solely of positively worded 
items. These- three factors have thp three highest Correlations between 
them. Belligerence is composed solely of negative, items anri' its three 
correlations with social skills, friends nnd nood are much reduced fro-n 
where they should be (by comparison with the correlations for older 
children found in our earlier research). Compliance and self-concept 
are each half positive *nd half negative in content. 

Thus th£ correlations in the whole sample ha^« a structure which is the 
composite of two structures superimposed on one another. One 
underlying pattern is the correlations between clusters as determine^ 
by item content. The other pattern is the item direction artifact: 
positive correlations between items in the same direction and negative 
correlations between items inapposite directions. One explanation for 
a composite structure is to assume that there are two kinds*of 
children: those who understand the instructions and respond to the 
items, and those who misunderstand the instructions and respond to all 
items alike. If some of the children who misunderstand the 
instructions tend to always say "always" while some of them always say 
"never", then among- the children who misunderstand there would be a 
tendency for all items to be perfectly correlated in their original 
form. However, after the negative items are reverse scored, the all 
positive correlations matrix would turn into one with the item 
direction artifact. Pooling the data across the two types of children 
produces a correlation matrix for the whole sample that is a comoosite 
of the correlation matrices for the two groups. This was our , ~~ 
hypothesis/ to explain the results found in the exploratory and 
confirmatory factor an^lysps. 

Identifying Res ponse Biases 

*"* 1 * " ' , * 

If all ,the children who misunderstood the instructions either answered 
all the items "always" or Always answered "never", then the bad data 
would be easy to detect. However, it was likely that these children 
would not follow any pattern perfectly. Therefore, the following 
decision rule was developed* ' 

We first listed the children v/ho had /answered over half of all it^ms 
"always" or Had answered over half of all items "never". This produced 
a list of 21 children. One child had answered ever^item '-'always" and 
another child had answered all but one item "neveN^ M\ oth^r-cas^s 
were imperfect.* However, in all cases, the children had either said 
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"always" to more than half of both the positive and to the native 
items. Thus we felt no qualms ahout % discarding these cases as bad d*t£. 

f /e then computed a new correlations matrix and carried out all analyses 
on it. The results were as we had predicted: the item direction 
artifact v/as 'reduced and the data looked much more "like the data for 
the older children; however, a strong item direction Artifact 
remained. The final decision rule is shown in figure?." 

Our specific criteria were: (1) The average* score of both positive and 
negative items both were required*to* be greater than 2 when the total 
test average gcore was greater tffen 2.3. or (both averages were 
required to be less than 2 wherrthe overall average was less than 1.7. 



AVERAGE ON 
NEGATIVE HEMS 




NEVER SO^ETiMES ALWAYS AVERAGE ON 

POSITIVE ITEMS 



Figure 1. Graphic dlslay of decision rule for bad data, shading 
indicates >.he region of probable bad data. 

The central box in Fi^urp 1 represents the set of pll possible pairs of 
average responses across positively and negatively worded items. That 
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is f since items are scored 1 t 2 f or 3; the average must be between 1 
and 3 for each set of items. The top righft corner represents the 
perfect all "always" response set. The bottom left corner represents . 
the perfect all "never" response set*. The shaded areas represent the 
cases near these corners that were rejected by our decision rule. 

By using this decisiofrsoiieS we discarded ^ cases out of «05 t yielding 
a discard rate of about 10 percent. 

Exploratory Factor Analysis - Reduced Sample 

In order to test the hypothesis that the factor structure sho#n in 
Table % was due to response bias rather than item content, we re^gn « 
the analysis for the students selected by the decision rule described 
above. The pattern, shown in Table *i is strikingly different: 3« out 
of 10 of the-rregative items loaded on Factor 1 t while 7 of 13 of the 
positive itens loaded on Factor II. There was still some correlation 
between item direction and factor loading, but then th^re is also a 
certain amount of correlation built into the content structure as 
well.* tfe take the results in Table H to be a confirmation of our 
hypothesis that the item direction artifact in this data is due to a 
response bias for a portion of the younger children. This again 
suggests the need for a better instructional procedure prior to testing. 

Table U. Factor loadings and item direction for the 3*0 children left 
after 45 cases wejjy? dropped because they showed evidence of , 
failure to understand the instructions. Contingency table 
relating the item direction to thp hi<tfi factor loading for 
that factor. 

Item 
Direction 

* High + 

Factor t n 3 

Loading II „ 7 7 



Confirmatory factor analysis* of the reduced sample 

- After the *J5 cases with suspect data were discarded we computed a new 
correlation matrix. The factor analytic results have been partially 
reported in Table U; these data showed a drastic reduction in- the item 

p direction artifact. This section reports the results relevant to the 
confirmatory factor analysis. 

A correlation matrix for the reduced sample was developed with the 
[ items grouped according to the 8 clusters of the content analysis, 
i This matrix is included in Appendix D. This arrangement showed th*t 

the bad items of the whole sample are still bad items. Thss item ?n 
did not correlate, with the other social skill items, and items 2? and 
\ '25 did not correlate with the other mood items. The correlation 
between looks items increased, but only to .0* t and the correlations 
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between the "smart" /items remained low as well, ranging from -.01 to 
.20. Therefore, tftk final recommendation for the cluster structure 
remained the sapie and a residual 'duster was formed with the items from 
the smart and>Iooks clusters and items 20*, -1, 23, 



Table 5 shpwn an exploratory factor analysis of the reduced sanole. 
Four principal components were defined without communal i ties* The 
b f5 le i J ^ arr f n 5ed with the refined clusters grouped together down the 

W y ' each iten withln a given cluster* would have exactly the 

same pattern of facto;* loadings as each other item in that cluster. 
This was approximately true for the six main clusters with the bad 
items eliminated. * « - f 



Table 5. Exploratory factor analysis of the reduced sample done with 
comnunalities (decimals omitted, M = 360, highest loading 
marked with an asterisk, items grouped by refined clusters). 
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"A confirmatory factor analysis was completed- on the refined clusters * < 

for the reduced sample. This analysis .showed a good- fit to the 
confirmatory model for all six of the main clusters* The matrix from 
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this analysis is included in Appendix E," 
t 

Table 6< showsjjhe correlation between factors in the confirmatory 
-factor analysis,. The correlations were all substantially less than 
1.00 f showiflTg that the factors Were all well separated statistically. 
The distorting- effeets of item direction were reduced but still 
present; as ^especially .seen in the high correlations between social 
skills,, friends and mood. 

- ¥ • 

The analysis of the reduced sample tends to confirm the hypothesis that 
thene was a substantial amqjjfit of bad data due to student 
misunderstanding of the instructions. However, it is also obvious tjhat 
not all the bad data were found. Thi*| it JLs hard to know what credence 
can be given to the present findings in- regard to the correlation 
between' factors. 



Table 6. The factor correlations for the defined clusters on the 
reduced sample (decimals omitted, M = 350). 



Social Skills 

Belligerence 

Compliance 

Friends 

Setf-Concept 

Mood 



ss 


BEL 


C0H ■ 


FEI 


SC 


MOOD 


100 


61 


45 


~6~T 


46 


-58, 


64 


100- 


59 


25 


20 


'15 


45 


59 


100 


. 30 


■ 45^ 


35 


61 • 


25 


30 


100 


* 3?/ 


62 


46 


. 20 


45 


38 


100 


37 


58 


45 


35 




" 37 


100 



Recommendations for fostrurfr^nt Improvement 




The administration of* the questionnaire requires more of an emphasis on 
teaching children how to take the test. First pf all.'ther* should be 
a pre-test using items intended to be "wasted". Ideally there would be 
a set of items with correct answers that could be checked to see if the 
child understands the instructions. Those children who do not could be 
individually tutore'd. • 

The items from the looks and smart clusters should' be discarded {alon* 
-with items 20, 3, 23 and 25). These 11 items' could be replaced by * - 
others tapping the mhtn clusters. Since item Reliability is low, these 
clusters badly need to be increased in length. ' / 

If more items than 28 are needed, the test could be given over more 
than one day or in two sessions on one day/ In fact for reliability 
purposes, and. for increased power %in item analysis, the same test could 
be given with a gap of a week or two. Item responses could then be 
averaged across sessions to generate "super-items", i.e., versions of 
the items with about twice the reliability. 

The emphasis on positive items in the current questionnaire may be 
raisDlaced* If there is an asymmetry, in response to items at either end 
of the scale ♦ then it is the negative items which should be stressed 
since it is the negative end which to be assessed* 

/ * -* 
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1. Sometimes we feel like we can do everything real well, like we are » 
the best in everything. How often do you; feel you can do every- 
thing well? Circle, the A box-for always, the,S "triangle for sometimes 
the N circle far never, 

-|°!i e t™es we feeL.really dumb, JiJcejffi.can'ib figure -out anythinq - 

. How often do you feel dumb? Circle the A box for always, the S 
triangle for sometimes and the N circle for never. 

3. Sometimes we feel really good-looking. How often do you^feel you 

. really look good? Circle the A box if you feel you look good all- 
the time, the S triangle if you feel you look good sometimes, and 
the N circle if you never feel you look good. 

4. How many of the kid? in this class like you? Circle the A box if 

• -' a !'°t ^ em llke y° u » the ? triangle, if some of them like you, the N 
circle if none of them like you. 

5. How often are you proud of the work you do in school? Circle the 
A box if you're always proud of your work, the S triangle you're 
sometimes proud of your work, and the N circle if you are never 
proud of your work. 

6. How often does. your teacher feel you do good work in class? Circle 
. the A box. if (she/he) always feels you do good work, the S triangle 

if (she/he) sometimes feels you do- good work, and the N circle if 
(she/he) never feels you'do good work. 

7. Sometimes our friends really like our ideas, How often do your 

friends like your ideas? Circle the'A box if they always like your 

. ideas, the -5 traangle if they sometimes like them and N circle if 
^they never 1-ike them. * 

8. How many of the kids in this class don't like you? Circle the' A box 
• . J.f. a11 of them d on't like .you, the S triangle 'if some of them don't 

like you and the H circle if none of them don't Like ypu. 
* . - • . * 

9. Sometimes we feel kind of ugly - like no one would want 'to look at 
1 . '' us - How often do you feel like that? Circle the A box if you 

always feel ugly, the S triangle if you Sometimes feel ugly, and 
. the N circle if you never ^feel ugly. 

10. How many kids in this class would want to sit next to you. Circle 
the A box if all of them would, the S triangle if some of them 

> would, and-the N circle if none of them would. 

11. Sometimes we don't like ourselves. How often do you feel that way' 
Circle # the A box' for all of the time, the S triangle for some of the 
time, and. the N circle^for none of the time. . 

* • 

12. Do you feel like you are an important person in your class? Circle 
the A box for always, the S triangle for sometimes, and the N 
circle for never. ' 




13. When some kids get really angry at another la'4, they hit them. How 
often do you hit people yau get mad at? . Qircle the A box if you 
always hit theft, the S triangle'if you sometimes hit them, and the 
N' circle if you* never hit them. 

14. When some kids don't like other kids in class, they tease them or 
—^—calHthenr names. Hovroften xio you. tease the ktds you don't like? ~ 

Circle the A box if you always tease the kids you don't like, the 
S triangle If^you sometimes tease the kids you don't like, and the 
N circle if you never tease the kids you don't like, 

15. How often d# you pay attention in class? Circle the A box if you 
always pay attention, the S triangle if you sometimes pay attention, 
and the N circle if you never gay attention. 

16. How often do you act up in class? Circle the A box if you always 
act up in clatas, circle the S triangle if you sometimes act up in 
class, and circle the N xircle^if you never act up in class. 

17. How often do you try hard when you do your school work? Circle the 
A box if you alwa/s try hard, the S triangle if you sometimes try' 
hard, and the N circle if you never try hard. 

18. Sometimes kids stop doing what they are supposed to do when the 
teacher leaves the room. How often' do you do that? Circle the A 
box if you always stop doing what you're supposed to, the 3* triangle 
if you sometimes stop doing what you're suppose^ to, and the \X 
circle if you never stop doing what you're supposed' to. ♦ 

'19. When you get a special "treat in your lunch, how often do you share 
it? Circle ..the A box "if you always share it, the S triangle if you 
sometimes 'share it, and the N circle, if .you never share it. 

20.. Pretend that there was a kid in class that most of your friends 
didn't like. Would you, let that kid play with you at recess? 
Circle the A box for always, the S triangle for sometimes, the N 
circle for never. > 



21. When you see that another kid in class is really sad about something, 
how often do you try to cheer them, up? Circle_the A box if you 
always try .to cheer them up, the S triangle if you sometiems try to . 

^ cheer-^them up, and the N circle if you never try to cheer them up. 

22. How often do you do something special for your friends, just to make 
them feel good? Circle, the A box if you doMt a lot of the time, 
the S triangle if you do it some of the time, and the N circle if 
you never do it. / 

23. 4 Sometimes we feel that there is nobody we can talk to. How often 

do you^feel that way? Circle the A box if you always feel 'that 
way, the S triangle if you sometimes fee/I that way, and the N circle 
; if if you never feel that way. 

ERIC • . 58 . 
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24. How often do you really have fun when you are in school? Circle the'-* 
- A box if you always have fun, the S triangle if you sometimes have r 

fun, and the N circle if you never-' have fun. ^ 

25. Sometimes people get scared that bad things will happen, scared that 
people' won't Vi ke them, or that things will go wrong* or that they 
wflT get* in trouble. How often are you scared that bad things will 
happen? Circle the A box if you are scared a lot of the time,, the S 
triangle if you are scared some of the time, and the N circle if you 
are never scafed bad things will happen..- 

26. Sometimes we *are happy and sometimes we are sad. How much of the 
time are you happy? Circle the A box if you're always happy, the S 
triangle if you're sometimes happy, and the N circle if you're never 

, happy. 

Sometimes, when really good things happen, we feel really, really 
"great. inside. We want to, tell everyone" about it, we feel excited. 
How often do, you feel like that? Circle the A box if you feel that 
way a lot of the time, the S triangle if you feel that way some of 
the time, and the H circle if you never feel that way. 

28. Sometimes we get so interested in something that we want to do it all 
the time. How often do you get that interested in the things you do? 
Circle the A box if you're interested a lot of the time,, the S triangle 
if you're interested some 1 of the time, and the N circle if you're * 
never interested, in what you're doing. 
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APPEHOIX B 
THE ITEM CORRELATION - ENTIRE SAMPLE 
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The item correlations for the entire sample grouped 
according to the original content clusters (decimals 
omitted, H = 405). 
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APPENDtX C 



Confirmatory factor analysis on the refined clusters 
for the whole sample (decimals omitted, m H * 405). 
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The item correlations on the reduced sample with items 
grouped according to the original clusters determined 
by the content analysis (decimals omitted, H s 360). 
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CONFIRMATORY FACTOR ANALYSIS - R EDUCEn Sa>'?LE 
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APPENDIX £ 

Confirmatory factor analysis on the refined clusters 
for the reduced sample (decisis ositted, H • 360, 
comma! ities used). 



501 SOCIAL SKILLS (19, 21, 22) 

502 BELLIGERENCE (13, 14} 

503 COMPLIANCE (15, 17, 16, 18) 

504 FRIENDS (4, 10) 

505 SELF CONCEPT (1, 11) 

506 MOOD (24, 26, 27, 28) 

507 RESIDUAL (3, 9. 2, 5, 6, 7, 12, '20, 25, 8, 23) 
STANDARD SCORE COEFFICIENT ALPHAS 

41. 40. 48. 32. 31. 55. 38. 
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